Methods and systems for detecting a tailgating event at an access point controlled by an access control system

ABSTRACT

Apparatus and methods for controlling access to a restricted area by an access control device includes obtaining a first image using a first sensor mounted at a first location. A second image is obtained using a second sensor mounted at a location different from a location of the first sensor. The second image is processed, using the second sensor, to obtain information regarding the detected objects in the second image. The information regarding the detected objects is sent from the second sensor to the first sensor. The first sensor compares the received information with a number of objects detected using the first image. A tailgating event is identified, in response to determining that the number of objects detected using the first image does not match the information regarding the number of objects detected using the second image. A tailgating notification is outputted, by the first sensor, indicating a tailgating event.

TECHNICAL FIELD

The present disclosure generally relates to access control systemsimplementing visual recognition, and more specifically, to detecting atailgating event at an access point controlled by an access controlsystem.

BACKGROUND

An access control system is often used at public and private premises,such as households, commercial buildings, businesses, retailestablishments, schools, hospitals and government buildings, to list afew examples. Access control system nodes may be installed at accesspoints of the premises (such as, but not limited to, external andinterior doors of a building) to control access to restricted areas,such as the building itself or to areas within the building. The accesscontrol system authenticates (or authorizes) an individual and thenpermits the authenticated/authorized individual to access the restrictedareas through the access point.

Historically, the main components of the access control system areaccess input devices (such as but not limited to card readers, keypads,cameras, biometric input devices) and access point controllers (such as,but not limited to, a device that controls a position of the accesspoint (e.g., open/closed) or a state of the access point (e.g.,locked/unlocked). The access input device is typically installed at anentry area of the access points and enables receipt of one or morecredentials (e.g., access card, security code, biometric information) toobtain access to the restricted areas. If the individual is identifiedand is authorized to obtain access to the restricted area, then theaccess input device may signal the access point controller, for example,to unlock a door and/or to not generate an alarm. However, in such asystem, there is a possibility of unobserved or uncontrolled tailgating,where another individual following closely behind the authorizedindividual is able to access the restricted area without beingauthorized.

In view of the foregoing, there is a need to more efficiently and moresecurely, and in more user friendly manner, control access to restrictedareas.

SUMMARY

The following presents a simplified summary of one or moreimplementations of the present disclosure in order to provide a basicunderstanding of such implementations. This summary is not an extensiveoverview of all contemplated implementations, and is intended to neitheridentify key or critical elements of all implementations nor delineatethe scope of any or all implementations. Its sole purpose is to presentsome concepts of one or more implementations of the present disclosurein a simplified form as a prelude to the more detailed description thatis presented later.

The present disclosure relates to an access control system thatimplements visual recognition to identify and authenticate an individualto control access to a restricted area. In some cases, the accesscontrol system may further operate in conjunction with multiple visualsensors to determine how many individuals are attempting to enter therestricted area.

One example implementation relates to a method controlling access to arestricted area by an access control device. One aspect of the methodcomprises obtaining a first image of a surveillance area adjacent to anaccess point to the restricted area using a first camera mounted at afirst location. The first sensor is configured to detect a number ofobjects in the first image of the surveillance area. Another aspect ofthe method comprises obtaining a second image using a second sensormounted at a location different from a location of the first sensor. Thesecond sensor is communicatively coupled to the first sensor. The secondsensor is configured to detect one or more objects in the second image.In another aspect of the method, the second image is processed, usingthe second sensor, to obtain information regarding the one or moredetected objects in the second image of the surveillance area. Theinformation regarding the one or more detected objects is sent from theto the first sensor. Another aspect of the method comprises comparing,by the first sensor, the information regarding the one or more objectsdetected using the second image with a number of objects detected usingthe first image; and identifying, by the first sensor, a tailgatingevent in response to determining that the number of objects detectedusing the first image does not match the information regarding thenumber of objects detected using the second image. In another aspect ofthe method, a tailgating notification is outputted, by the first sensor,indicating a tailgating event, in response to identifying the tailgatingevent.

Additional advantages and novel features relating to implementations ofthe present disclosure will be set forth in part in the description thatfollows, and in part will become more apparent to those skilled in theart upon examination of the following or upon learning by practicethereof.

DESCRIPTION OF THE FIGURES

The novel features believed to be characteristic of the disclosure areset forth in the appended claims. In the descriptions that follow, likeparts are marked throughout the specification and drawings with the samenumerals, respectively. The drawing figures are not necessarily drawn toscale and certain figures may be shown in exaggerated or generalizedform in the interest of clarity and conciseness. The disclosure itself,however, as well as a preferred mode of use, further objects andadvances thereof, will be best understood by reference to the followingdetailed description of illustrative aspects of the disclosure when readin conjunction with the accompanying drawings, wherein:

FIG. 1 illustrates a schematic view of an example operating environmentof an access control system implementing detection of a tailgatingevent, in accordance with aspects of the present disclosure;

FIG. 2 is a schematic view of another example operating environment ofan access control system implementing detection of a tailgating eventsimilar to FIG. 1 but further including detection of tailgating indifferent regions of a surveillance area, in accordance with aspects ofthe present disclosure;

FIG. 3 is a flowchart of an example method of operation, andinteractions between various modules of, the access control systemimplementing tailgating event detection, in accordance with aspects ofthe present disclosure;

FIG. 4 is a front view of an example visual and audible feedback accesscontrol device in accordance with aspects of the present disclosure; and

FIG. 5 is a block diagram of various hardware components and otherfeatures of a computer system that may operate the access control systemin accordance with aspects of the present disclosure.

DETAILED DESCRIPTION

This disclosure relates to systems, devices, and methods for managingand controlling access to a restricted area by specific users, such asauthorized and/or unauthorized individuals attempting to access therestricted area, by determining whether a number of authorizedindividuals in a surveillance area adjacent to an access point to therestricted area as detected by a first sensor device matches with anumber of individuals in the surveillance area detected by a secondsensor device. In particular, in the present disclosure, the systemutilizes at least the first and second sensor devices, such cameras, forobtaining different images of a surveillance area from different anglesand a management system for determining the access permission level ofthe identified individuals and for detecting potential tailgating eventswhen the number of individuals in the different images does not match.

Referring to FIG. 1 , in an example implementation, an operatingenvironment 100 includes an access control system 102 for managing andcontrolling access, via an access point 107, to one or more restrictedlocations 105 by one or more specific users 101, by determining amismatch between a number of authorized users 101 in a surveillance area103 identified by a first sensor 106A and a number of individuals, e.g.,authorized user 101 and individual 109, identified by one more othersensors, for instance, sensors 106B and 106C. The access control system102 may comprise a plurality of sensor(s) 106A-106C, such as cameras,that each obtain an image of the surveillance area 103, which mayinclude one or more user(s), such as authorized user 101 and/or one ormore other unauthorized users, such as individual 109. In one aspect,the images generated by the sensors 106A-106C may be a video comprisinga plurality of image frames, or a single still frame. For example, inone example implementation, a first sensor 106A may be located on a wallnear an access point, such as a door to a restricted area. In someimplementations, the plurality of sensors 106A-106C may comprise aplurality of camera devices used in conjunction with each other. Forexample, the first sensor device 106A may capture an image of a user 101comprising their face as they approach the location of the access point107, while the second sensor device 106B and/or the third sensor device106C, which may be located on a ceiling, or on a different position onthe wall or on a different wall relative to the first sensor device106A, may supplement the image obtained by the first sensor device 106Awith an image of an area around the user 101. In this case, due to thedifferent perspectives of the second sensor device 106B and/or the thirdsensor device 106C relative to the surveillance area 103 and the firstsensor device 106A, the images captured may include one or more areasthat may not be within the field of view of the first sensor device 106Aand/or one or more areas within the field of view of the first sensordevice 106A but that may be obstructed by the user 101 or another personor object. In this case, for example, the second sensor device 106Band/or the third sensor device 106C may identify a second individual,e.g., individual 109, in the surveillance area 103 that was not detectedby the first sensor device 106A. As such, the supplemental informationobtained by the second sensor device 106B and/or the third sensor mayhelp the access control system 102 to determine whether more than oneuser is attempting to gain access to location 105, and if suchadditional users are not authorized, for indicating a tailgating event.

In an implementation, in response to obtaining an image or images, thesensors 106A-106C may transmit the raw data of the image(s) to a controlunit 108 via bus 122. In an implementation, bus 122 may be a CAT6shielded cable or any other suitable bus that provides both datatransfer and power to the sensors 106A-106C. In an implementation, thesecond sensor 106B and the third sensor 106C may be communicativelycoupled to the first sensor 106A via bus 122. In another exampleimplementation, the sensors 106B-106C may have object detectioncapabilities described in greater detail below. In such implementation,the second sensor 106B and the third sensor 106C may send informationabout the detected object(s) (object detection results) to the firstsensor 106A via bus 122.

The control unit 108 receives the raw data from the sensors 106A-106C,and may forward the raw data via bus 126 to an Object IdentificationServer 110. In another example implementation, the control unit 108 maysupport wired or wireless communication (e.g., Bluetooth™ and/or WiFi),which can allow for multifactor authentication, when implemented inconjunction with facial recognition. Further, bus 126 may also beconnected a network switch 124 (e.g., a POE+ switch), allowing broadbandcapability for remote access.

The Object Identification Server (OIS) 110 may receive the raw livevideo and/or the still image(s) from one or more sensors 106A-106C forprocessing to detect/identify/classify objects and to determine anidentity(s) of the user(s). The OIS 110 may perform detection of theobject using the received video and using deep learning algorithms (forexample, You Only Look Once (YOLO) algorithm). The detection performedby the OIS 110 will be referred to as object detection. An object regioncan include a region bounded by a boundary of the object. In someimplementations, the OIS 110 may output the detection result of anobject (also referred to as an “object detection result” or a “detectionresult” hereinafter). An object detection result may include informationfor each object, for example, information indicating the position of theobject, the size of the object, and the like. In some implementations,the object detection result may include, for example, informationindicating a rectangle circumscribing a region in a frame of a videofrom which each object is detected, the coordinate values of thecentroid of the object region, information indicating the width of eachobject, information indicating the height of each object, and the like.In some implementations, the object detection result may include otherinformation. For example, the object detection result may include thecoordinate values of the uppermost end and the lowermost end of theobject region instead of or in addition to the coordinate values of thecentroid of the object region. The object detection result may alsoinclude information for each object such as information indicating theposition and size of each object and/or approximate distance betweenobjects, if more than one object is detected by the OIS 110.

The OIS 110 may also perform facial recognition on the raw streamobtained from the first sensor 106A. Further, in another exampleimplementation, the OIS 110 may store digital representations of objectsand facial data entered previously, not actual images of objects/faces.Examples of digital representations of object data and facial data mayinclude, but are not limited to, bitmaps, hashed or other encodedversions of the image data, etc. The OIS 110 may be updated with new oradditional digital representations of objects/users via managementserver 112. The OIS 110 may be located in the same location as thesensors 106A-106C and control unit 108, for example, the same buildingor room, or may be located in a remote location, for example, in thecloud or a remote building from the management server 112 and thesensors 106A-106C. In one example of operation, the OIS 110 may comparethe obtained facial recognition of the raw data stream to the digitalrepresentations of faces previously stored in a database to determinewhether there is a match. In response to the comparison, the OIS 110 maygenerate an identification determination signal indicating if theuser(s) is/are known (e.g., a match with the previously entered facialdata is found or determined within a threshold level of confidence) orunknown, and providing information about the identification if theperson is known (e.g., a name and/or identifier associated with theidentified user). In another example implementation, the OIS 10 may notgenerate the identification determination signal if the user(s) is/areunknown in the database. In another example implementation, thegenerated identification determination signal may further comprise aconfidence score. The OIS 110 may transmit the generated identificationdetermination signal to the control unit 108 and/or the managementserver 112 via bus 126.

The management server 112, described in detail below, may also receivethe identification determination signal via the bus 126. The managementserver 112 may log/record the identification determination signal; forexample, the comparison results may be logged/recorded, the confidencescore may be logged/recorded, the time and/or date the comparison wasperformed may be logged/recorded, the decision by the OIS 110 may belogged and/or recorded, and/or the name/identity of the user may belogged/recorded. The management server 112 may be located in the samelocation as the sensors 106A-106C and control unit 108, for example, thesame building or room, or may be located in a remote location, forexample, the cloud or a remote building from the management server 112and the sensors 106A-106C.

Referring back to the control unit 108, the OIS 110 may transmit theidentification determination signal via bus 126 to the control unit 108,which may operate to control access to the restricted area. The controlunit 108 may transmit the identification determination signal to anaccess device 114 via bus 130. In one example implementation, thecontrol unit 108 may support Bluetooth and/or WiFi communications, whichmay allow other types of multifactor authentication. In one exampleimplementation, the control unit 108 may transmit an indication signalback to the first sensor 106A via bus 122. The first sensor 106A mayoutput indications to the user at the location of the first sensor 106Avia lights and/or sound. The visual or light-based indications willdiscussed in detail below. In one example implementation, control unit108 may exchange messages with access device 114 via bus 130, whereinsuch communication messages may be in a particular protocol, forexample, Open Supervised Device Protocol (OSDP) and Wiegand.

The access device 114 may store access control information, which mayinclude data that identifies if a user is allowed access to a specificlocation in order to make an access control decision. For example, theaccess device 114 may store information indicating that a known user,e.g., user A, is permitted to access a specific location. Further, theaccess control information may be stored as a lookup table comprisinginformation on specific users and the specific locations for which theymay be allowed access.

In an optional aspect, the management server 112 may supply the accessdata stored in the access device 114 via bus 128. The access data may beupdated on a specific schedule, for example, every day, or may beupdated when new information is placed in the management server 112. Inanother example implementation, the access data may be stored solely onthe management server 112, or shared between the management server 112and the access device 114.

In any case, for example, the access device 114 may receive theidentification determination signal via bus 130 that indicates that user101 is recognized and identified, as described above. The access device114 may attempt to determine if user 101 has access to restrictedlocation 105 via the access data stored in the device, or request thedata from the management server 112.

Upon determination to grant or deny access to an identified user, anaccess control signal is transmitted via bus 132 and 130 indicating theaccess status of the identified user.

The control unit 108 may receive the access control signal from theaccess device 114, and the information from the one or more additionalsensors, e.g., second and/or third sensor devices 106B and/or 106C, inorder to generate an output signal to enable access or to provide atailgating notification. For example, the control unit 108 may receivethe access control signal via bus 130 comprising the access statusindicating if the identified user has been granted access or deniedaccess to a specific location. Further, the control unit 108 may receiveadditional camera info as a continuous metadata stream (of XML formatteddata) across an IP network connection. The metadata will contain detailsof the objects identified in the video stream along with their locationand size.

Additionally, the control unit 108 may compare a number of authorizedusers identified via the first sensor device 106A to a number ofindividuals identified by one or more additional sensors, e.g., secondand/or third sensor devices 106B and/or 106C, respectively. In otherwords, the control unit 108 may validate the tailgating event prior togenerating the output signal.

In one example implementation, the control unit 108 may transmit theoutput signal back to the first sensor 106A via bus 122. The firstsensor 106A may output indications to the user at the location of thefirst sensor 106A via lights and/or sound based on the output signal.The visual indications and audible indications will be discussed indetail below, but may indicate that access is allowed or denied, and/orthat a tailgating event is detected.

The access control system 102 may further include a controllable lockingunit 116, for example attached to a door hinge, frame, door knob,handle, lock, or another feature or structure of the access point 107that is removably blocking access to the location 105. The locking unit116 may receive the output signal, via bus 132, where the output signalincludes the access status indicating if the identified user has beengranted access or denied access to the location 105 and/or thetailgating notification. For example, the locking unit 116 may be anelectronically controllable mechanical actuator that can be controlledto move between a locked position and an unlocked position.

In one example implementation, if the user(s) has been identified, asdescribed above, and has been granted access to the location 105 via thecontrol device 114 and a tailgating event has not be identified, thenthe locking unit 116 unlocks the access point 107, such as but notlimited to a door (or a gate, etc.) to enable the user 101 to access thelocation 105. In an example implementation, an access point sensor 118may be used to determine if the access point 107 has been opened,closed, how long the access point 107 has been opened and how long theaccess point 107 has been closed, in order to make subsequentaccess-related decisions. For example, the locking unit 116 may notunlock the access point 107 again if the access point 107 has not yetbeen determined to be closed via access point sensor 118. Further, anemergency opening device 120 may be present on the reverse side of theaccess point 107. The emergency opening device 120 may be triggered by auser 101 in an emergency to immediately unlock the access point 107 viathe locking unit 116.

In another example implementation, if the user(s) has been identifiedand has been granted access to the location 105 but a tailgating eventis identified, then the locking unit 116 does not unlock the accesspoint 107 until the tailgating event is resolved.

Further details regarding an example method of operation of the systemwith respect to detecting tailgating events are illustrated in FIG. 2and will be described below with reference to FIG. 3 .

The structure and configuration of the first sensor 106A is described indetail below with reference to FIG. 4 .

Referring to FIG. 2 , in another example implementation, the operatingenvironment 200 includes all or portions of the access control system(shown in FIG. 1 ) for managing and controlling access to specificlocations by specific users, by determining the identities of usersbased upon their visual appearance and by detecting tailgating events,according to an aspect of the disclosure. As described above withreference to FIG. 1 , the same aspects are illustrated in FIG. 2 . Inone aspect, the first sensor 106A may further interact with one or moreadditional sensor(s) 106B, such as additional video camera(s), todetermine if tailgating users are present in order to issue a tailgatingnotification or to allow access to the restricted area. The additionalsensor(s), such as the second sensor 106B, may all be placed atdifferent locations and/or at different installation heights. In oneexample implementation, rather than using facial recognition alone, anumber of users in a surveillance area may be further identified basedon the images captured from the second sensor 106B. In addition, thesecond sensor 106B may capture additional user information, for example,their physical proportions, and/or other non-biometric data particularto the user. Working in conjunction with each other, sensors 106A and106B may provide improved user validation and may provide detection oftailgating event for the access control system 102. For instance, sincefirst sensor device 106A may be installed at head height, it might beunable to identify one or more individuals 109 standing/walking behindthe individual 101 located closest to the first sensor 106A. This is anevent referred to hereinafter as a tailgating event.

In an aspect, the second sensor 106B may be configured to providebi-directional counting of people, by detecting objects entering/exitinga scene from user marked regions. In an aspect, people counts can bereported to the first sensor device 106A, for example. Count data may besaved directly on-board the second sensor 106B up to a predefined numberof days. In addition, the second sensor 106B may alarm the first sensor106A when objects enter into a user-drawn zone from outside of the zone.For example, the second sensor 106B may alarm the first sensor 106A whenobjects move from a predefined amber zone 202 to a predefined red zone204.

In an aspect, if more than one object, e.g., user or individual such asuser 101, is detected within a monitored zone (such as zones 202 and204, which may comprise surveillance area 103 described in FIG. 1 ), thesecond sensor 106B, for example, may be configured to determine adistance between the detected objects. For example, the second sensor106B may utilize interferometry, time-of-flight, and/ortriangulation-based principles depending on the requirements onaccuracy, the standoff distance between the sensor and the object, andthe required depth of field.

In an aspect, if more than one sensor is used to monitor thesurveillance area 103, each sensor may perform its own approximation ofthe distance between the detected objects and may send the approximatedvalues to the control unit 108. In an aspect, the control unit 108 maycollect approximations from all sensors and may calculate theapproximate distance between the detected objects based on thecumulative data received from all sensors 106A-106C.

In an aspect, calibration techniques may allow for the calibration of anarbitrary number of sensors 106A-106C in the operating environment 200.Calibration may involve determining some or all of a variety ofparameters and coefficients. For example, calibration may solve for oneor more intrinsic parameters such as focal length and image center. Asanother example, calibration may determine one or more distortioncoefficients such as models of radial and tangential distortion. As yetanother example, calibration may solve for one or more extrinsicparameters defining a position of an object in a scene relative to apattern or other sensors that identified the same pattern in a scene.

In some examples, calibration may be performed at least in part by usinga calibration pattern, which may be a known set of features in 2D or 3D.For instance, a known pattern of dots may be used, where the distancebetween each dot and the other dots is known. Calibration may beperformed at least in part by collecting multiple different views of anobject. In further examples, capturing multiple views of a calibrationpattern in different positions may allow for (1) calibration of the oneor more coefficients of the camera and/or (2) knowledge of where thecamera is relative to the coordinate system established by where thecalibration pattern was fixed.

Each camera 106A-106C has an associated point of view and field of view.A point of view refers to the position and perspective from which aphysical region is being viewed by a camera. A field of view refers tothe physical region imaged in frames by the camera.

As shown in FIG. 2 , during initial setup phase, the first sensor 106Amay send analytics configuration information 206 to the second sensor106B. This configuration information may define corresponding zones202-204 to be monitored by each camera 106. The configurationinformation 206 may further include metadata type, for example. Eachcamera 106A-106C may associate metadata 208 with images of the movingobject (referred to as an “object” for short). These metadata 208 definevarious characteristics of the object. For instance, the metadata 208can define the location of the object within the camera's field of view,the width or height of the image of the object (e.g., measured inpixels), the direction the image of the object is moving in, the speedof the image of the object, the color of the object, and/or a categoryof object. These are pieces of information that can be present inmetadata associated with images of the object; other metadata is alsopossible. The category of object refers to a category, based on variouscharacteristics of the object. For example, categories can include:humans or animals. Metadata regarding events involving moving objects208 may also be transmitted by the second sensor 106B to the firstsensor 106A. Such event metadata 208 may include: an object entering thefield of view of the camera, an object leaving the field of view of thecamera, the object remaining in the camera's field of view for greaterthan a threshold period of time, multiple moving objects detection,object counting, distance between objects, an object entering an area ofinterest (e.g., a predefined area where the movement of objects isdesired to be monitored, such as red zone 204), an object leaving apredefined zone, an object moving in a direction matching a predefinedforbidden direction for a zone, object removal, object abandonment(e.g., when an object is still longer than a predefined period of timeand its size is smaller than a large portion of a predefined zone), anda dwell timer (e.g., the object is still or moves very little in apredefined zone for longer than a specified dwell time). In an aspect,the metadata 208 may include a distance between the detected objects.The metadata 208 may also indicate whether any of the detected objectsare wearing a mask.

The control unit 108 (shown in FIG. 1 ) may analyze the live raw videoreceived from each of the sensors 106A-106C and provide an outputcontaining the identified user or users in the live video. The outputmay further include an indicator of a determined tailgating event and aconfidence level of such determination. In one example implementation,the control unit 108 may also store the video, the location and/or timethat a tailgating event is identified as the historical data. In oneexample implementation, when tailgating determination is not complete,for example, partially obscured or from a longer distance away, thecontrol unit 108 may still provide tailgating information with aconfidence score based on a previous, higher confidence determination incombination with matching one or more characteristics associated withthe previous determination, e.g., a particular size, shape, and/or colorof an article of clothing. The method of the system illustrated in FIG.2 will be described below with reference to FIG. 3 .

Referring to FIG. 3 , an example of a method 300 of the operation of andinteractions between various modules of the access control systemimplementing tailgating event detection is explained in accordance withone implementation. FIGS. 1 and 2 may be referenced in combination withthe flowchart of FIG. 3 . To start, method 300 includes one or moreusers attempting to access a specific access control location, asdescribed above. At step 302, the first sensor 106A, e.g., a camera, maygenerate the raw live video stream (a first image of the surveillancearea). In an aspect, the first sensor 106A may transmit the raw videosteam to the control unit 108. The first sensor 106A may be mounted at afirst location. In an aspect, the first sensor 106A may contain aplurality of segments of a light ring, as described below with referenceto FIG. 4 . The first sensor 106A may be configured to perform facialrecognition of the user attempting to access the restricted area. Atstep 304, the second sensor 106B may also generate the raw live videostream (a second image of the surveillance area) In an aspect, thesecond sensor 106B may also transmit the raw video steam to the controlunit 108. The second sensor 106B may be mounted at a second locationdifferent from the first location of the first sensor and at heightdifferent from the mounting height of the first sensor 106A. The secondsensor 106B may be communicatively coupled to the first sensor 106A viabus 122. The second sensor 106B may be configured to process thegenerated video in order to detect and count one or more objects (e.g.,individuals) in the second image of the surveillance area (step 306).The second sensor 106B may send metadata 208 (e.g., the informationregarding the one or more detected objects to the first sensor) at step308. The first sensor 106A may compare the information regarding theobjects detected by the second sensor 106B with a number of objectsdetected using the first image at step 310. More specifically, this isthe number of individuals that are identified (or not known) via facialrecognition.

If the number of person count (a count of number of individuals in thesecond image) from second sensor 106B and the facial recognition count(a number of individuals in the first image having authorized access tothe restricted area) from first sensor 106A does not match (e.g., thesecond count exceeds the first count) (decision at 311, no branch) andat least one of the numbers is greater than one, then at step 316 asignal is sent indicating a detected tailgating event. Alternatively, ifthe number of detected objects does match (decision at 311, yes branch),then an output signal is generated at 315 that confirms that there isnot a tailgating event, and access may be granted if the one or moreobjects (e.g., individuals) are authorized to have access via the accesspoint.

In an optional aspect, at steps 312 and 314, the control unit 108 mayconfirm the tailgating event by comparing the video generated by bothsensors 102A-102B. If the tailgating event is detected and/or confirmed(decision at 314, yes branch), a signal may be sent to the first sensor106A at step 316. Alternatively, if the tailgating event is notconfirmed (decision at 314, no branch), e.g., if the number of detectedindividuals by the first sensor 106A and the second sensor 106B isdetermined to match, then an output signal is generated at 315 thatconfirms that there is not a tailgating event, and access may be grantedif the one or more objects (e.g., individuals) are authorized to haveaccess via the access point.

When the tailgating event is determined, the first sensor 106A mayreceive the signal sent at step 316 and may be configured to provide avisual and/or audible notification to the user at the access location,as described below. If the tailgating event is detected and all usersare identified, a signal may be sent to the first sensor 106A and theaccess device 114 at step 316. The first sensor 106 may receive thesignal sent at step 316 and may be configured to provide a visual and/oraudible notification to the user at the access location, as describedbelow. Further, the access device 114 receives the signal transmittedduring step 316. If at least one user associated with the detectedtailgating event is denied access, the locking unit 116 does not unlockthe door. If all the users associated with the detected tailgating eventare identified and granted access, a signal may be sent to the lockingunit 116, in response to which the locking unit 116 unlocks the door atthe access point.

In an aspect, the control unit 108 may determine user notificationcriteria. For example, the control unit 108 may determine theillumination sequence of the first sensor 106A, described below, basedon at least a signal indicating tailgating event determination, user(s)identification, access determination and face determination.Alternatively, the control unit 108 may ask the first sensor 106A torender an audible notification. Such an audible indication of thetailgating event could be a recording saying, for example, “More thanone person is detected, please move away from each other.” The controlunit 108 may send the criteria to the first sensor 106A at step 316.

At step 318, the first sensor 106A may render the notification. Forexample, the first sensor 106A may illuminate the LEDs in a specificillumination sequence based on the system determinations, describedbelow.

In an aspect the method 300 may return to step 302 after generating thenotification in order to allow the individuals to space themselves apartin response to the notification and retry the access procedure.

In other words, the method 300 includes a method for controlling accessto a restricted area by an access control device. The method includesobtaining a first image of a surveillance area adjacent to an accesspoint to the restricted area using a first sensor mounted at a firstlocation. The first sensor is configured to detect a number of objectsin the first image of the surveillance area. A second image is obtainedusing a second sensor mounted at a location different from a location ofthe first sensor. The second sensor is communicatively coupled to thefirst sensor. The second sensor is configured to detect one or moreobjects in the second image. The second image is processed, using thesecond sensor, to obtain information regarding the one or more detectedobjects in the second image of the surveillance area. The informationregarding the one or more detected objects is sent from the secondsensor to the first sensor. The information regarding the one or moreobjects detected using the second image is compared with a number ofobjects detected using the first image by the first sensor. A tailgatingevent is identified, by the first sensor, in response to determiningthat the number of objects detected using the first image does not matchthe information regarding the number of objects detected using thesecond sensor. A tailgating notification is outputted, by the firstsensor, indicating a tailgating event, in response to identifying thetailgating event.

In one or any combination of these aspects, the information regardingthe one or more detected objects in the second image includes at least acount of individuals detected in the surveillance area and anapproximate distance between the individuals detected in thesurveillance area.

In one or any combination of these aspects, the first sensor sends thefirst image to the access control device. The second sensor sends thesecond image to the access control device. The access control deviceconfirms the tailgating event by comparing the first image with thesecond image. The access control device generates a control signal basedon the confirmation. The control signal includes an access deniedcommand when the tailgating event is confirmed.

In one or any combination of these aspects, a light emitting device ofthe first sensor is illuminated to provide an access indication basedupon the control signal. The access indication includes a first patternof illumination when the control signal indicates the detectedtailgating event. The access indication includes a second pattern ofillumination when the control signal does not indicate the detectedtailgating event.

In one or any combination of these aspects, each of the firstilluminated pattern and the second illuminated pattern is produced in aplurality of colors.

In one or any combination of these aspects, an audible output isgenerated using a speaker of the first sensor based upon the controlsignal.

In one or any combination of these aspects, the first sensor sends tothe second sensor, prior to obtaining the second image, configurationinformation including at least information identifying the surveillancearea.

In one or any combination of these aspects, the first location is at aheight lower than the second location.

In one or any combination of these aspects, obtaining the first image ofthe surveillance area using the first sensor and obtaining the secondimage of the surveillance area using the second sensor occur within atime duration threshold.

Similarly and in other words, from the perspective of the first sensor,the method 300 includes obtaining a first image of a surveillance areaadjacent to an access point to the restricted area using a first sensor,mounted at a first location. The first sensor is configured to detect anumber of objects in the first image of the surveillance area. The firstsensor receives information regarding the one or more detected objectsin a second image obtained by the second sensor. The second sensor ismounted at a second location different from the first location of thefirst sensor. The second sensor is communicatively coupled to the firstsensor. The information regarding the one or more objects detected usingthe second image is compared with a number of objects detected using thefirst image by the first sensor. A tailgating event is identified, bythe first sensor, in response to determining that the number of objectsdetected using the first image does not match the information regardingthe number of objects detected using the second sensor. A tailgatingnotification is outputted, by the first sensor, indicating a tailgatingevent, in response to identifying the tailgating event.

Referring to FIG. 4 , an example visual feedback access control deviceincludes the first sensor 106A, as referenced above, which may contain aplurality of segments of a light ring 406. Each of the plurality ofsegments of the light ring 406 may comprise a plurality of lightemitting diodes (LEDs). The LEDs, for example, may be a single LEDdevice containing one red, green, and blue (RGB) LEDs. In one exampleimplementation, the first sensor 106A of FIG. 4 may comprise 8 segments,each of the 8 segments of the light ring 406 may each contain at leastone LED. In one example implementation the first sensor 106A maycomprise a plurality of sensor devices 402. As described above, the oneor more sensor devices 402 may be a video camera, or a still camera thatcaptures a plurality of images. In another example implementation, thefirst sensor 106A may alternatively or additionally comprise a pluralityof infrared (IR) illumination units 404. The IR illumination units 404may detect infrared energy (heat) and convert the detected heat into anelectronic signal, which is then processed to produce a thermal image.In another example implementation, the IR units 404 may aid in nighttime illumination and also provide illumination in the day time toimprove the facial data gathered for recognition. As illustrated in FIG.4 , for example, 16 separate IR illumination units are present, but thisimplementation is not limiting and it should be understood that anynumber of IR illumination units may be present.

In one example implementation the first sensor 106A may also comprise aspeaker and a microphone (not shown). The speaker and the microphone maybe configured to work as an intercom systems. In an implementation, forexample, the speaker and microphone may communicate with othercomponents of the system using a communication protocol, such as but notlimited to the Session Initiation Protocol (SIP). In one exampleimplementation, the speaker may be configured to generate tailgatingnotifications when multiple users attempt to access a specific location.The first sensor 106A may provide additional audible notifications tousers, for example, that their image has been captured, the user hasbeen identified, the user has been granted access to the specificlocation, the user has been denied access, the user has not beenidentified, and the like. For example, the audible notifications may beimplemented with a synthesized voice or play an audio file to the user.Bus 122 in FIGS. 1 and 2 , as described above, my provide a signal togenerate the notifications. In another example implementation, themicrophone may provide audible communications to employees that manageoperation of the system, for example, a two-way communication betweenthe user 106 and an employee operating the system to discuss reasons forthe user's denial of access to a specific location, requestingadditional information from the user(s) associated with the detectedtailgating event, or the like. In another example implementation thespeaker and/or microphone of the first sensor 106A may provide audiblenotifications and communication regarding an alarm or duress situation.

As described above, the first sensor 106A may receive signals via bus122 referenced in FIGS. 1-3 . The signals may instruct the first sensor106A to illuminate the segments in different illumination sequencesbased on one of a plurality of trigger events. For example, some triggerevents may be: a tailgating event detected by the first sensor 106A; theimage being a match with one of a plurality of known images in adatabase when compared by the OIS 110; the image not being the matchwith any of the plurality of known images in the database of the 110;one or more users wearing a mask; or an alarm/duress event. Theillumination sequences generated by the LEDs, for example, may berotating green and blue colors; solid colors such as all yellow, allblue, all green all red; only the top half with solid colors; only thebottom half with solid colors; or flashing colors. Any color may beassociated with any illumination sequence and with any trigger event.For example, each trigger event may have a respective pre-programmedillumination sequence, or each trigger event may be user selected.

Aspects of the present disclosure may be implemented using hardware,software, or a combination thereof and may be implemented in one or morecomputer systems or other processing systems. In one aspect, thedisclosure is directed toward one or more computer systems capable ofcarrying out the functionality described herein. FIG. 5 presents anexample system diagram of various hardware components and other featuresthat may be used in accordance with aspects of the present disclosure.Aspects of the present disclosure may be implemented using hardware,software, or a combination thereof and may be implemented in one or morecomputer systems or other processing systems. In one example variation,aspects of the disclosure are directed toward one or more computersystems capable of carrying out the functionality described herein. Anexample of such a computer system 500 is shown in FIG. 5 .

Computer system 500 includes one or more processors, such as processor504. The processor 504 is connected to a communication infrastructure506 (e.g., a communications bus, cross-over bar, or network). Varioussoftware aspects are described in terms of this example computer system.After reading this description, it will become apparent to a personskilled in the relevant art(s) how to implement aspects of thedisclosure using other computer systems and/or architectures.

Processor 504, or any other “processor,” as used herein, processessignals and performs general computing and arithmetic functions. Signalsprocessed by the processor may include digital signals, data signals,computer instructions, processor instructions, messages, a bit, a bitstream, or other computing that may be received, transmitted and/ordetected.

Communication infrastructure 506, such as a bus (or any other use of“bus” herein, such as bus 122, 126, 128, 130, and/or 132 as discussedabove), refers to an interconnected architecture that is operablyconnected to transfer data between computer components within a singularor multiple systems. The bus may be a memory bus, a memory controller, aperipheral bus, an external bus, a crossbar switch, and/or a local bus,among others. The bus may also be a bus that interconnects componentsinside a access control system using protocols, such as Controller Areanetwork (CAN), Local Interconnect Network (LIN), Wiegand and OpenSupervised Device Protocol (OSDP) among others.

Further, the connection between components of computer system 500, orany other type of connection between computer-related componentsdescribed herein may be referred to an operable connection, and mayinclude a connection by which entities are operably connected, such thatsignals, physical communications, and/or logical communications may besent and/or received. An operable connection may include a physicalinterface, a data interface and/or an electrical interface.

Computer system 500 may include a display interface 502 that forwardsgraphics, text, and other data from the communication infrastructure 506(or from a frame buffer not shown) for display on a display unit 530.Computer system 500 also includes a main memory 508, preferably randomaccess memory (RAM), and may also include a secondary memory 510. Thesecondary memory 510 may include, for example, a hard disk drive 512and/or a removable storage drive 514, representing a floppy disk drive,a magnetic tape drive, an optical disk drive, etc. The removable storagedrive 514 reads from and/or writes to a removable storage unit 518 in awell-known manner. Removable storage unit 518, represents a floppy disk,magnetic tape, optical disk, etc., which is read by and written toremovable storage drive 514. As will be appreciated, the removablestorage unit 518 includes a computer usable storage medium having storedtherein computer software and/or data.

In alternative aspects, secondary memory 510 may include other similardevices for allowing computer programs or other instructions to beloaded into computer system 500. Such devices may include, for example,a removable storage unit 522 and an interface 520. Examples of such mayinclude a program cartridge and cartridge interface (such as that foundin video game devices), a removable memory chip (such as an erasableprogrammable read only memory (EPROM), or programmable read only memory(PROM)) and associated socket, and other removable storage units 522 andinterfaces 520, which allow software and data to be transferred from theremovable storage unit 522 to computer system 500.

It should be understood that a memory, as used herein may includevolatile memory and/or non-volatile memory. Non-volatile memory mayinclude, for example, ROM (read only memory), PROM (programmable readonly memory), EPROM (erasable PROM) and EEPROM (electrically erasablePROM). Volatile memory may include, for example, RAM (random accessmemory), synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM(SDRAM), double data rate SDRAM (DDR SDRAM), and/or direct RAM bus RAM(DRRAM).

Computer system 500 may also include a communications interface 524.Communications interface 524 allows software and data to be transferredbetween computer system 500 and external devices. Examples ofcommunications interface 524 may include a modem, a network interface(such as an Ethernet card), a communications port, a Personal ComputerMemory Card International Association (PCMCIA) slot and card, etc.Software and data transferred via communications interface 524 are inthe form of signals 528, which may be electronic, electromagnetic,optical or other signals capable of being received by communicationsinterface 524. These signals 528 are provided to communicationsinterface 524 via a communications path (e.g., channel) 526. This path526 carries signals 528 and may be implemented using wire or cable,fiber optics, a telephone line, a cellular link, a radio frequency (RF)link and/or other communications channels. In this document, the terms“computer program medium” and “computer usable medium” are used to refergenerally to media such as a removable storage drive 514, a hard diskinstalled in hard disk drive 512, and signals 528. These computerprogram products provide software to the computer system 500. Aspects ofthe disclosure are directed to such computer program products.

Computer programs (also referred to as computer control logic) arestored in main memory 508 and/or secondary memory 510. Computer programsmay also be received via communications interface 524. Such computerprograms, when executed, enable the computer system 500 to performvarious features in accordance with aspects of the present disclosure,as discussed herein. In particular, the computer programs, whenexecuted, enable the processor 504 to perform such features.Accordingly, such computer programs represent controllers of thecomputer system 500.

In variations where aspects of the disclosure are implemented usingsoftware, the software may be stored in a computer program product andloaded into computer system 500 using removable storage drive 514, harddrive 512, or communications interface 520. The control logic(software), when executed by the processor 504, causes the processor 504to perform the functions in accordance with aspects of the disclosure asdescribed herein. In another variation, aspects are implementedprimarily in hardware using, for example, hardware components, such asapplication specific integrated circuits (ASICs). Implementation of thehardware state machine so as to perform the functions described hereinwill be apparent to persons skilled in the relevant art(s).

In yet another example variation, aspects of the disclosure areimplemented using a combination of both hardware and software.

The aspects of the disclosure discussed herein may also be described andimplemented in the context of computer-readable storage medium storingcomputer-executable instructions. Computer-readable storage mediaincludes computer storage media and communication media. For example,flash memory drives, digital versatile discs (DVDs), compact discs(CDs), floppy disks, and tape cassettes. Computer-readable storage mediamay include volatile and nonvolatile, removable and non-removable mediaimplemented in any method or technology for storage of information suchas computer readable instructions, data structures, modules or otherdata.

It will be appreciated that various implementations of theabove-disclosed and other features and functions, or alternatives orvarieties thereof, may be desirably combined into many other differentsystems or applications. Also that various presently unforeseen orunanticipated alternatives, modifications, variations, or improvementstherein may be subsequently made by those skilled in the art which arealso intended to be encompassed by the following claims.

What is claimed is:
 1. A method for controlling access to a restrictedarea by an access control device, comprising: obtaining a first image ofa surveillance area adjacent to an access point to the restricted areausing a first sensor, mounted at a first location, wherein the firstsensor is configured to detect a number of objects in the first image ofthe surveillance area; obtaining a second image of the surveillance areausing a second sensor mounted at a second location different from thefirst location of the first sensor, wherein the second sensor iscommunicatively coupled to the and wherein the second sensor isconfigured to detect one or more objects in the second image of thesurveillance area; processing the second image, using the second sensor,to obtain information regarding the one or more detected objects in thesecond image of the surveillance area; sending the information regardingthe one or more detected objects in the second image from the secondsensor to the first sensor; comparing, by the first sensor, theinformation regarding the one or more detected objects in the secondimage with the number of objects detected using the first image;identifying, by the first sensor, a tailgating event in response todetermining that the number of objects detected using the first imagedoes not match the information regarding the one or more objectsdetected using the second image; and outputting a tailgatingnotification, by the first sensor, to indicate a tailgating event, inresponse to identifying the tailgating event.
 2. The method of claim 1,wherein the information regarding the one or more detected objects inthe second image comprises at least a count of individuals detected inthe surveillance area and an approximate distance between theindividuals detected in the surveillance area.
 3. The method of claim 1,further comprising: sending, by the first sensor, the first image to theaccess control device; sending, by the second sensor, the second imageto the access control device; confirming, by the access control device,the tailgating event by comparing the first image with the second image;and generating a control signal based on the confirmation, wherein thecontrol signal comprises an access denied command when the tailgatingevent is confirmed.
 4. The method of claim 3, wherein outputting thetailgating notification includes: illuminating a light emitting deviceof the first sensor to provide an access indication based upon thecontrol signal, wherein the access indication comprises a first patternof illumination when the control signal indicates the detectedtailgating event, and wherein the access indication comprises a secondpattern of illumination when the control signal does not indicate thedetected tailgating event.
 5. The method of claim 4, wherein each of thefirst illuminated pattern and the second illuminated pattern is producedin a plurality of colors.
 6. The method of claim 3, wherein outputtingthe tailgating notification includes: generating an audible output usinga speaker of the first sensor based upon the control signal.
 7. Themethod of claim 1, further comprising sending, by the first sensor tothe second sensor, prior to obtaining the second image, configurationinformation comprising at least information identifying the surveillancearea.
 8. The method of claim 1, wherein the first location is at aheight lower than the second location.
 9. The method of claim 1, whereinobtaining the first image of the surveillance area using the firstsensor and obtaining the second image of the surveillance area using thesecond sensor occur within a time duration threshold.
 10. The method ofclaim 1, wherein identifying the tailgating event comprises determiningthat the number of objects detected using the first image is less thanthe one or more objects detected using the second image.
 11. The methodof claim 1, further comprising: wherein the number of objects in thefirst image of the surveillance area comprises a first count of a numberof individuals in the first image having authorized access to therestricted area; wherein the information regarding the one or moredetected objects in the second image of the surveillance area comprisesa second count of number of individuals in the second image; and whereinidentifying the tailgating event comprises determining that the secondcount exceeds the first count.
 12. A method for controlling access to arestricted area by an access control device, comprising: obtaining afirst image of a surveillance area adjacent to an access point to therestricted area using a first sensor, mounted at a first location,wherein the first sensor is configured to detect a number of objects inthe first image of the surveillance area; receiving, by the firstsensor, information regarding the one or more detected objects in asecond image obtained by the second sensor, wherein the second sensor ismounted at a second location different from the first location of thefirst sensor and wherein the second sensor is communicatively coupled tothe first sensor; comparing, by the first sensor, the informationregarding the one or more detected objects in the second image with thenumber of objects detected using the first image; identifying, by thefirst sensor, a tailgating event in response to determining that thenumber of objects detected using the first image does not match theinformation regarding the one or more objects detected using the secondimage; and outputting a tailgating notification, by the first sensor, toindicate a tailgating event, in response to identifying the tailgatingevent.
 13. A system for controlling access to a restricted area by anaccess control device, comprising: a hardware processor configured to:obtain a first image of a surveillance area adjacent to an access pointto the restricted area using a first sensor, mounted at a firstlocation, wherein the first sensor is configured to detect a number ofobjects in the first image of the surveillance area; obtain a secondimage of the surveillance area using a second sensor mounted at a secondlocation different from the first location of the first sensor, whereinthe second sensor is communicatively coupled to the first sensor andwherein the second sensor is configured to detect one or more objects inthe second image of the surveillance area; process the second image,using the second sensor, to obtain information regarding the one or moredetected objects in the second image of the surveillance area; send theinformation regarding the one or more detected objects in the secondimage from the second sensor to the first sensor; compare, by the firstsensor, the information regarding the one or more detected objects inthe second image with the number of objects detected using the firstimage; identify, by the first sensor, a tailgating event in response todetermining that the number of objects detected using the first imagedoes not match the information regarding the one or more objectsdetected using the second image; and output a tailgating notification,by the first sensor, to indicate a tailgating event, in response toidentifying the tailgating event.
 14. The system of claim 13, whereinthe information regarding the one or more detected objects in the secondimage comprises at least a count of individuals detected in thesurveillance area and an approximate distance between the individualsdetected in the surveillance area.
 15. The system of claim 13, whereinthe hardware processor is further configured to: send, by the firstsensor, the first image to the access control device; send, by thesecond sensor, the second image to the access control device; confirm,by the access control device, the tailgating event by comparing thefirst image with the second image; and generate a control signal basedon the confirmation, wherein the control signal comprises an accessdenied command when the tailgating event is confirmed.
 16. The system ofclaim 15, wherein the hardware processor configured to output thetailgating notification is further configured to: illuminate a lightemitting device of the first sensor to provide an access indicationbased upon the control signal, wherein the access indication comprises afirst pattern of illumination when the control signal indicates thedetected tailgating event, and wherein the access indication comprises asecond pattern of illumination when the control signal does not indicatethe detected tailgating event.
 17. The system of claim 16, wherein eachof the first illuminated pattern and the second illuminated pattern isproduced in a plurality of colors.
 18. The system of claim 15, whereinthe hardware processor configured to output the tailgating notificationis further configured to: generate an audible output using a speaker ofthe first sensor based upon the control signal.
 19. The system of claim13, wherein the hardware processor is further configured to send, by thefirst sensor to the second sensor, prior to obtaining the second image,configuration information comprising at least information identifyingthe surveillance area.
 20. The system of claim 13, wherein the firstlocation is at a height lower than the second location.